Model: allenai/OLMo-7B-0424-hf

Layer: 5. Neuron Index: 10602

gate+_in+

gate+_in-

gate-_in+

gate-_in-

Frequency: 0.84%. Frequency: 0.43%. Frequency: 47.32%. Frequency: 51.41%.
hook_post:
Max: 1.56;
Min: 0.00;
Avg: 0.18.
hook_post:
Max: 0.00;
Min: -0.68;
Avg: -0.03.
hook_post:
Max: 0.00;
Min: -0.48;
Avg: -0.05.
hook_post:
Max: 0.39;
Min: 0.00;
Avg: 0.05.
hook_pre_linear:
Max: 2.52;
Min: 0.00;
Avg: 0.93.
hook_pre_linear:
Max: 0.00;
Min: -1.82;
Avg: -0.31.
hook_pre_linear:
Max: 1.96;
Min: 0.00;
Avg: 0.24.
hook_pre_linear:
Max: 0.00;
Min: -1.77;
Avg: -0.24.
hook_pre:
Max: 1.09;
Min: 0.00;
Avg: 0.20.
hook_pre:
Max: 0.98;
Min: 0.00;
Avg: 0.12.
hook_pre:
Max: 0.00;
Min: -1.77;
Avg: -0.61.
hook_pre:
Max: 0.00;
Min: -1.73;
Avg: -0.63.
swish:
Max: 0.82;
Min: 0.00;
Avg: 0.11.
swish:
Max: 0.71;
Min: 0.00;
Avg: 0.07.
swish:
Max: 0.00;
Min: -0.28;
Avg: -0.20.
swish:
Max: 0.00;
Min: -0.28;
Avg: -0.21.

Prototypical activations for case gate+_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15




Prototypical activations for case gate+_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15




Prototypical activations for case gate-_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15



Prototypical activations for case gate-_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15


Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15